This invention relates to the field of face recognition, and, in particular, to face-identification for law-enforcement and corrections applications.
Law enforcement organizations rely heavily on witnesses to a crime for purposes of identifying criminals based on pictures of the criminal. Typically, a witness will review pages and pages of xe2x80x9cmug shotsxe2x80x9d of criminals from books of mug shots maintained by the law enforcement organization. The witness hopefully recognizes the criminal from the mug shots.
Over the years, huge collections of mug shots have been developed by local, state and federal law enforcement agencies. Some of these collections of mug shots have been deposited in a national depository for mug shots maintained by the Federal Bureau of Investigation (FBI) of the U.S. Department of Justice. It is estimated that in the United States there are approximately 60 million mug shots in various mug shot collections across the country. However, the FBI has only about one-third of all U.S. mug shots in its own collection.
Mug shots are typically a frontal and profile views of an individual. A mug shot usually comprises two photographs of the front and side view of the face of an individual. To provide standardization of mug shots, law enforcement photographers follow specific guidelines on the positioning of the front view and side views of a person for photographing mug shots. Accordingly, each pair of mug shots, i.e., the front face and side face views, is uniform with respect to angle of view, distance and lighting of the person being photographed. These mug shots have been stored in photographic books, on film, on tape and now digitally on computer discs.
Mug shots show only two views of an individualxe2x80x94the front face view and a profile face view. To recognize an individual suspect from mug shots, a witness must recognize the front face or profile view of the individual. However, the witness may not have seen the suspect from the front or profile view. Rather, the witness may have seen an angled view of the suspect""s face, such as from above, below or to one side. The view that the witness sees at a crime scene is often different than the front or profile view shown in mug shots. For a witness to recognize a suspect by reviewing the mug shots, the witness must convert in his own mind the angle from which he saw the suspect to either or both a front face or side view of the individual as shown in the mug shots. Not all witnesses can reliably convert in their mind the view actually seen of the suspect to the front face or side views shown in mug shots. Because all witnesses cannot make this conversion, witnesses from time to time have not been able to identify suspects. Moreover, witnesses at criminal trials are subject to cross-examination about how they recognized a suspect based on the frontal or profile view shown in a mug shot when the witness never saw the suspect from those particular views. Accordingly, criminals have not been identified or have been acquitted at trial because the witness did not reliably recognize the criminal suspect when reviewing mug shots, or because the witness was effectively cross-examined at trial with respect to the view that he had of the suspect.
There are many other facial features that are more difficult to describe and identify, such as the size of a person""s nose, description of their face, etc. These other facial features are often recalled and described by witnesses to a crime, and can be used to recognize a suspect. However, these other facial features are not easily categorized and, thus, have prevented the creation of an effective system for categorizing or indexing mug shots. In the past, law enforcement investigators have attempted to characterize facial features. For example, law enforcement organizations have created standard measurement tools for identifying the size of person""s eyes, nose, mouth, facial structure and ears. The criteria that are used are primarily used for identifying mug shots are limited 2-D criteria based on a full frontal view or profile view of a face. For example, the criteria as to nose width and eye shape may be limited to just front view figures, and not to profile views. Similarly, ear shape criteria may be applicable only to profile views and not to frontal view pictures of the individual. Thus, these criteria are of limited use at best if the witness observed an individual from a perspective other than a frontal or profile perspective.
Prior to the present invention, no known techniques were used for indexing facial features based on three-dimensional (3-D) framework. Prior approaches to cataloging and indexing facial features have assumed that the faces are shown only in 2-D full frontal and/or profiled views such as in mug shots. These indices of profile and frontal views of mug shots are helpful in identifying suspects when a front face or profile face view is available of the suspect. For example, if the suspect looks straight into a security camera such that there is a front face image, then that front face image can be indexed using standard face indices. Using the indices of the security camera picture of the suspect, the indicie values can be used to identify mug shot pictures having the same indicie values in searching for the criminal. Similarly, if a witness working with a law enforcement sketch artist can generate a frontal face and/or profile view of a suspect, that frontal face and profile sketch can be used to categorize the facial values of the criminal. These facial values can also be used to access the mug shots which are by the facial values indices used to obtain those mug shots having the same facial value indices.
The indices of facial features used to index mug shots are not useful when a frontal face or profile picture or sketch of the perpetrator is unavailable. Side views of a suspect cannot be readily categorized by facial indexing used with the current mug shots. A side view picture of a criminal is not susceptible to identifying the facial feature values used with current indices of mug shots. Unfortunately, the majority of photographs taken of suspects by security cameras are of angled views of the suspect""s face other than frontal face or profile. Thus, when a security camera captures on film a suspect, often that picture cannot be directly used to identify the suspect by mug shots.
It has been long recognized to be an undesirable trait that the mug shot indices do not allow for correlation with pictures of criminals other than straight front or straight profile views. It is a long-felt need in law enforcement to have a computerized system for recognizing faces in which a photograph or other 2-D image of a criminal from any view can be quickly and reliably analyzed and compared to a mug shot collection to select those mug shots of possible perpetrators of a crime.
The present invention is a three-dimensional (3-D) face-identification system useful for human faces and other image identification applications. The invention also provides a fast search engine for searching a large collection of two-dimensional mug shot photographs for those that match a particular facial image. In addition, the invention provides a technique for indexing 2-D mug shots to a uniform set of 3-D facial feature parts, such that a 3-D face surface images are generated based on the 2-D mug shots. Moreover, the invention provides a method by which a large number of mug shots may be reproduced using 3-D face surfaces by reference only to the uniform set of 3-D facial surface features and a composite code that identifies the facial feature parts to be used to generate a particular individual""s facial image. This method avoids the need to locally store a large amount of data that would be required if every mug shot were stored locally. In addition, the invention has the advantage of generating a 3-D facial image from any desired point of view, and of providing a searchable code to 2-D mug shot photographs from a 3-D image reconstruction of a face.
The present invention provides three-dimensional face-identification system which incorporates facial fiducial values (points). These fiducial values are used to index a facial image determining the composite code for the image. The composite code is used to regenerate the facial image from a group of standard facial feature parts. The present invention allows facial fiducial values to be measured from any photograph, sketch or picture of an individual""s face from any viewpoint. For example, a photograph of the face of a suspect from the perspective of 45xc2x0 off to the side of the suspect. The photograph is analyzed to determine its facial fiducial values corresponding to a three-dimensional (3-D) model of the face. Once these facial fiducial values are obtained with respect to a three-dimensional model, they are used to determine the proper composite codes for the 3-D facial image. These composite codes correspond to a facial feature parts database. Thus, the suspect""s 3-D face can now be generated using only the composite code and facial feature parts. In addition, the composite codes correspond to composite codes that index the two-dimensional mug shot photos. Thus, existing mug shot directories can be quickly searched by finding mug shots that have similar composite codes to the composite code for the face of the suspect to select mug shots of possible perpetrators. The selected mug shots are then accessed, preferably by a computer system and displayed on a computer screen to the witness.
In addition, the invention is able to convert the two-dimensional mug shot photos to a three-dimensional image. This three-dimensional image (shown as a 2-D display image), can be rotated automatically or manually by the viewer to a viewpoint that corresponds to the view that the witness had of the suspect. Moreover, the facial image shown on a display can be rotated to the viewpoint corresponding to the viewpoint of a suspect taken by a security camera or other photographic or video device. In this manner, more reliable and faster identification of suspects can be accomplished using standard two-dimensional mug shots that are selected using three-dimensional imaging fiducial feature points and composite codes. The face-recognition technology of the invention is based on a completely three-dimensional (3-D) framework. This critically-important approach of the invention is conceptually different from all existing face-recognition technologies. The invention completely adheres to the fact that naturally 3-D surfaces, which include 3-D face (head) surfaces, cannot be compared in different angled-view pauses based on two-dimensional (2-D) planar images, including the 2-D face images as frontal-view and profile-view mug shots.
Therefore, the invention establishes its 3-D framework based on 3-D face (head) surfaces 102, 104 (FIG. 1), which are subsequently used to establish the face-feature surfaces of the face-feature parts. The invention uses the available laser-scanner data for 3-D head surfaces (e.g., data generated by CARD Laboratory of Wright Patterson Air Force Base, U.S. Air Force). The available data for the 3-D head surfaces is used to generate the required 3-D surface mesh (grid/panel) structures 202 which can be considered in high-resolution 204 (FIG. 2), intermediate resolution 302 (FIG. 3) and low-resolution 402 (FIG. 4) formats for rendering 3-D twenty-four (24)-bit color image of facial and head surfaces.
The invention uses data for 3-D head surfaces (i.e., for more than 2000 humans) to establish a face-feature-surface repository. The invention identifies on each available 3-D head surface, up to thirty-two (32) face-feature parts 502 and separates their associated 3-D face (head) feature surfaces 504. The invention associates an xe2x80x9cindex ixe2x80x9d to each face (head) feature part 502 (e.g., nose as i=1, upper-lip as i-2, lower-lip as i=3, chin as i=4, etc.) to establish the required computer bookkeeping or indexing procedure. The invention considers the 3-D head-surface data for each different head; and by separating the face (head)-feature parts 502 for each head surface image, a computer generates a complete set of different face (head) feature surfaces 504 to form a repositgory of face feature surfaces.
The invention considers up to two-hundred fifty-six (256) different face-feature surfaces for each one of the designated facefeature part (i). Therefore, for each of the designated face-feature parts xe2x80x9cixe2x80x9d (e.g., nose, upper lip, lower lip, chin, left eye, right eye, etc.), the invention establishes up to 256 face-feature surfaces for each part by using the available laser-scanned 3-D head-surface data of up to 2000 heads. Conventional geometrical clustering techniques are used to categorize up to 256 distinct face-feature surfaces 504 for each designated face-feature part xe2x80x9cixe2x80x9d, from up to 2000 face-feature surfaces obtained from the available laser-scanned 3-D head-surface data.
To each categorized, distinctly-different face-feature surface 504 of the designated face-feature 502 (part i), the invention assigns a face-feature-surface code xe2x80x9cjxe2x80x9d, which is associated with all of its known geometrical and surface-color/complexion characteristics. Therefore, for computer bookkeeping purposes, the invention establishes a completely 3-D framework, based on a set of xe2x80x9cindices ixe2x80x9d, which identify the designated face-feature parts, and subsets of xe2x80x9ccodes jxe2x80x9d, for the face-feature surfaces, such that for any specified face-feature index i and the face-feature-surface code j, the invention can provide a completely 3-D, color face-feature surface, which can be rendered graphically in the frontal view, left-profile view, right-profile view and in all possible angled-view for observation and analysis.
Based on the results of its geometrical clustering analysis, the invention considers sets of face-feature-surface codes j, which contain low-numbered codes, starting with one (1) for the most common face-feature surfaces, and high-numbered codes, ending with two-hundred fifty-five (255), for the least-common (generally most prominent) face-feature surfaces. Fewer than 256 face-feature surfaces may be used for each face-feature part. For example, sixteen (16) face feature surfaces may be used for each face-feature part.
The invention considers, currently sixty-four (64), or as required up to one-hundred twenty-eight (128) face attributes, which include the designated, up to 32, face-feature parts. FIGS. 5 and 6 show some exemplary face and head feature parts 502. The invention uses the additional face attributes for considering such features as: (a) additional geometrical information (e.g., relative distances between the face-feature surfaces) of the face-feature parts; (b) color, tone, complexion, texture conditions of the face surfaces; (c) facial hair; d) abnormal characteristics as scars, bone damage, etc., and (e) other features.
The invention maintains, for each face-feature surface (with code j) of each face-feature part (with index i), a 3-D high-resolution (FIG. 2) surface mesh (vertex/panel structure) with 24-bit color information (FIG. 8) for graphically rendering 3-D face (head) surfaces. In order to further simplify the geometrical comparisons of different face-feature surfaces, the invention considers, for each face-feature part, a set of selected fudicial points which represent readily-identifiable characteristics locations on its face-feature surface. As an example, for the nose part, as the face-feature part with index i one (1), the invention considers twenty-one (21) points, including the top point, bottom point, tip point, nostril-side points, top-flare points, half point, quarter points, etc. (FIG. (9). The sets of fiducial points collectively guarantee that two nose-surface with matching sets of fiducial points, also match geometrically, within the limits of human perception and machine intelligence. As another example, for the ear parts, i.e., left ear, with index i as thirteen (13) and right ear with index i as fourteen (14), it considers eight (8) points, including the tragion points, maximum-back point, maximum-outward point, etc. (FIG. 10), which allows the invention to incorporate the geometrically-important characteristics of the ears in its 3-D face-recognition technology.
The invention considers the use of fiducial points with two novel conceptual and implementation features, that are uniquely different from all existing 2-D face-recognition technologies. First, the invention considers the sets of fiducial points, in the sense as fiducial templates, for all the face-feature parts, in contrast to the whole 2-D face image (or slightly distorted 2-D face images), associated with conventional 2-D face-recognition technologies. Second, the invention considers completely 3-D positions for the fiducial points, in contrast to the 2-D positions used in conventional 2-D face-recognition technologies. Therefore, the invention, by considering the completely 3-D structure of the sets of fiducial points (i.e., fiducial template) can generate correctly all the 2-D structures of the fiducial points on planes normal to the direction of line of sight to the face (head) for all angled-view observations.
The invention considers up to 256 3-D face-feature surfaces, identified by their xe2x80x9ccodes jxe2x80x9d or each of the 64 face-feature parts, designated by its xe2x80x9cindex ixe2x80x9d. For each face-feature part, e.g., nose with i as 1 (FIG. 11), chin, with i as 4 (FIG. 12), left/right ears, with i as 13 and 14, respectively (FIG. 13), the framework of the invention provides all the associated 2-D images of the 3-D face-feature surface as observed from any specified view, including the frontal view, left/right-profile view, and any left/right- and upward/downward-angled view. Therefore, the 3-D framework of the invention, which is based on 3-D face (head)-feature surfaces, can completely eliminate the geometrical problems associated with angled-view observations, as encountered in currently-available 2-D face-recognition technologies.
The invention uses its 3-D framework, based on face-feature parts (e.g., nose, upper lip, lower lip, chin, left eye, right eye, etc.), with each face-feature part that can be represented by one of the up to 256 associated face-feature surfaces (FIGS. 5-10), to construct distinctly different 3-D face-composites which can be rendered, in 24-bit color, as 3-D face (head) surfaces. According to the invention, the bookkeeping required for constructing a 3-D face (head)-composite consists of up to sixty-four (64), face-feature parts, identified by their xe2x80x9cindices ixe2x80x9d and the associated 3-D face (head)-feature surface, identified by its xe2x80x9ccode jxe2x80x9d (FIG. 14). The critically-important consequence of the 3-D framework of the invention, is that, with the already-stored 3-D, 24-bit color graphical information for all the face (head)-feature surfaces (up to 256) for each of the face (head)-feature part (up to 32), the system only requires information as xe2x80x9ccode jxe2x80x9d integers, between 1-255, for each xe2x80x9cindex ixe2x80x9d, to be able to render in 3-D, 24-bit color format the complete face (head)-composite. Considering that each xe2x80x9ccode jxe2x80x9d, associated with a face (head)-feature surface represents an 8-bit, i.e., 1 Byte integer number (binary), and considering up to 64 (or 128) face-feature attributes, including the face (head)-feature surfaces (up to 32), the invention require the specification of a maximum of 64 (or 128) 1 Byte integers, with a maximum storage requirement of only 64 Bytes (or 128 Bytes) for each composite-face (head) code (FIG. 14). Therefore, the invention provides the capability to store up to 60 million mug shot sets, based on frontal view and profile-view 2-D face images, with less than 4.0 Gigabyte of storage.
The mug shot conversions functionality of the invention provides the procedure for converting a set of two mug shots, as 2-D face images, into a 3-D face (head)-composite (FIGS. 15 through 25). First, the procedure considers the appropriately-cropped and sized 2-D face image of the frontal-view mug shot. It initiates the matching, e.g., conversion procedure, by finding among the already-stored nose-part repository (xe2x80x9cindex ixe2x80x9d as 1) of the nose surfaces, and installing on a base head, the nose surface (e.g., xe2x80x9ccode jxe2x80x9d as 090) that most-closely resembles the nose on the 2-D frontal-view face-image (FIG. 15). It accomplishes the appropriate matching based on geometrical comparisons of the 2-D locations of the fiducial-point templates of the 3-D nose surfaces (FIG. 16).
Second, the procedure considers the appropriately-cropped and sized 2-D face image of the profile-view mug shot. It continues the conversion procedure by matching, independently, and installing on the base head, the nose surface (xe2x80x9ccode jxe2x80x9d as 090) which most-closely resembles the nose on the 2-D profile-view face image (FIG. 17). It accomplishes the appropriate matching based on the geometrical comparison of the 2-D locations of the fiducial-point templates of the 3-D nose surfaces (FIG. 18). Third, it considers the results of the two independent matching procedures and decides, based on minimum geometrical error considerations, the 3-D nose surface (xe2x80x9ccode jxe2x80x9d as 090) which most-closely resembles the nose in both 2-D frontal-view and profile-view face images (FIGS. 16 and 18).
The mug shot conversion functionality of the invention continues the matching procedure for all, up to 32, facial-feature parts, including, e.g., the left/right-ear parts (FIGS. 19 and 20), and systematically continues the assembly of the face (head)-composite, by considering both the frontal-view and profile-view observation of the partially-assembled face (head)-composites at different levels (FIGS. 21 and 22). Finally, for the two, i.e., frontal-view and profile-view, 2-D face images of the mug shot set, the mug shot-conversion procedure of the invention establishes the completely assembled 3-D face (head) composite (FIGS. 23 and 24).
Two consequences of the mug shot-conversion procedure of the invention is critically important. First, the final, completely-assembled 3-D face (head)-composite naturally provides geometrically accurate and realistic representations of the frontal-view and profile-view observations corresponding to the set of two mug shots. But more importantly, it can also provide geometrically accurate and realistic representations of any and all specified angled-view observations of the 3-D face (head) composite corresponding to the set of two mug shots. Therefore, first, the invention establishes a geometrically-accurate and realistic 3-D face (head)-composite corresponding to the original 2-D face images of the mug shot set, based on, importantly, appropriate assembly of the 3-D face surfaces (with xe2x80x9ccode jxe2x80x9d) of the face-feature parts (with xe2x80x9cindex ixe2x80x9d), which are maintained in its face-feature-part repositories. Second, the invention establishes the essential face (head) composite code which requires at most 64 Bytes.
The mug shot conversion functionality accomplishes the conversion of the available 2-D, pixel-intensity-dependent face-image data, associated with the original set of two, frontal-view and profile-view mug shots, with a face (head)-composite code of at most 64 Bytes (FIG. 14). Consequently, the invention provides the ultimate data-compression format for storing the required information for the mug shot records.
The general picture-conversion functionality of the invention provides the procedure for converting a single, angled-view (arbitrary) 2-D face image of a person, generally as an unknown person, into a 3-D face (head)-composite (FIGS. 27 through 31). The picture conversion functionality of the invention is based on the same procedure of the mug shot conversion functionality, with the exception that the procedure is not limited to considering only the frontal-view and profile-view 2-D face-images of the mug shots, but one or more angled-view 2-D face images obtained as captured pictures (generally obtained from surveillance equipment). The picture conversion procedure considers the appropriately-cropped and sized 2-D face image of the available angled-view picture. It initiates the conversion procedure by matching, i.e., finding among the already-stored nose-part repository (xe2x80x9cindex ixe2x80x9d as 1) of the nose surfaces, and installing on an appropriately-oriented base head, the nose surface (e.g., xe2x80x9ccode jxe2x80x9d as 090) that most-closely resembles the nose on the 2-D angled-view face-image (FIG. 27). Again, it accomplishes the appropriate matching based on geometrical comparisons of the 2-D locations of the fiducial-point templates of the 3-D nose surfaces (FIG. 28). An important aspect of the picture conversion functionality of the invention is that the procedure systematically considers different angled-view conditions of the fiducial points of the 3-D nose-template to match simultaneously both the angled-view conditions (e.g., 20 degree rotated, 15 degree upward, and 5 degree sidewise tilted) and the 3-D nose surface (e.g., xe2x80x9ccode jxe2x80x9d as 090).
Again, the picture-conversion functionality of the invention continues the matching procedure for all, up to 32, facial-feature parts, including, e.g., right-ear part (FIG. 29) and systematically continues the assembly of the face (head)-composite, by considering only the single partially assembled face (head)-composites at different levels (FIGS. 30 and 31). Finally, for the given 2-D angled-view face-image as the captured picture of a person, the picture-conversion procedure of the invention establishes the completely assembled 3-D face (head)-composite (FIG. 31).
The two consequences of the picture-conversion procedure of the invention are critically important. First, the final, completely-assembled 3-D face (head)-composite naturally provides geometrically accurate and realistic representation of the given angled-view observation of the picture of the person. Again, more importantly, it can also provide geometrically accurate and realistic representations of any and all specified angle-view observations of the 3-D face (head)-composite, corresponding to the given 2-D angled-view face-image of the picture.
Therefore, again, the invention establishes a geometrically accurate and realistic 3-D face (head)-composite corresponding to the original 2-D face-image of the picture, based on, importantly, appropriate assembly of the 3-D face-surfaces (with xe2x80x9ccode jxe2x80x9d) of the face-feature parts (with xe2x80x9cindex ixe2x80x9d) which are maintained in its face-feature-part repositories. Second, again, the invention establishes the essential face (head)-composite code which requires at most 64 Bytes of binary data storage (FIG. 14).
The invention also provides a novel 3-D composite-generator system (FIG. 32) which can be used to generate 3-D composites of perpetrators (or suspects) based on information provided by the victim(s) and/or witness(es). The completely 3-D composite-generation system is designed and constructed as a stand-alone utility (user interface) which can generate 3-D face (head)-composites under all angled-view observation and special lighting conditions. Therefore, the composite generator system (FIG. 32) of the invention allows the law-enforcement/corrections personnel to generate realistic 3-D face (head)-composite according to the conditions similar to the ones encountered by the victim(s) and/or witnesses. The 3-D composite-generator system automatically generates the essential face (head)-composite code which can be used on the fly for comparisons of the generated 3-D face (head)-composite with the available (already processed) mug shot records.
Some of the novel features of the present invention include:
The 3-D face-recognition technology is compatible with existing 2-D images of mug shots. The 3-D images may be projected onto a display as 2-D profile and frontal views of a 3-D face image. The 2-D images are also mapped onto 3-D surface facial feature parts to convert the 2-D images into 3-D images. In addition, the indexing system used in connection with the 3-D images is also applied to the existing 2-D images so that the 2-D images may be retrieved based on the index for the 3-D images.
The 3-D face-identification technology, in which 2-D mug shot photographs are mapped to 3-D facial feature parts, allows angled view pictures of individuals to be generated from the existing 2-D mug shots. Accordingly, a witness may view a suspect from an angled view picture, and is not limited to the frontal and profiled views of a suspect provided by standard mug shots.
The 3-D technology generates each 3-D human face or head from 3-D facial feature parts, e.g., chin, eyes, nose, mouth, etc. In one embodiment of the invention, the human face and head has been divided into sixty-four facial feature parts. Each part is a 3-D facial surface image of an anatomically significant feature of a face or head. There may be sixty-four (64) facial feature parts. When all 64 facial feature parts are assembled in a mosaic a complete 3-D human face is formed. For each facial feature part, a wide variety of different part shapes, (such as 256 different facial feature surface shapes) are available and stored as a 3-D facial feature image surface. For example, a nose facial feature part may have 256 different nose shapes that can be selected in generating a particular nose for a facial image. Virtually any human face can be generated from the 64 different facial feature parts, and the group of facial feature shapes for each part.
Each shape in a group of a facial feature part is ranked in order of its distinctiveness with respect to the other shapes in the same group. For example, the most distinctive noses, e.g., large protruding noses, having a high ranking, and small, non-descript noses having a low ranking. Highly distinctive facial features tend to be the most quickly recognized and the most long remembered by witnesses. By assigning a high ranking to distinctive facial feature parts, these quickly recognized and most long remembered are displayed first to speed the facial recognition process.
Each of the facial-features parts and their associated shapes are stored in local memory as high-spatial-resolution-3-D geometric surface shapes. There may also be color complexion data that is locally stored for use in applying skin color tones to the generated 3-D images of faces and heads. The amount of memory required to store the data for all of the 3-D 64 facial feature parts and the associated 256 part shapes is about two gigabytes of binary storage space. In contrast, the memory space required for the available forty million 2-D mug shots would require about 73,780 gigabytes, and even if these mug shots are compressed the memory requirement would be 734 gigabytes. It is substantially more practical to locally store on a law enforcement computer 3-D facial feature parts that can be assembled to form any face image, rather than to locally store the actual 2-D images.
With the present invention, the number of facial images is not a dominant factor determining the memory storage requirements. All human faces can be formed from the facial feature parts stored locally to a law enforcement computer. An image of any particular individual may be generated from the facial feature parts stored in the computer. There are identifying codes (i, j composite codes discussed below) for those individual""s whose faces have been cataloged into a mug shot database. The identifying codes are used to identify the features of the individual""s face. Moreover, the identifying codes are correlated to the 3-D facial feature parts. The memory required to store the identifying codes of an individual is negligible as compared to the memory space required to store an actual image of the individual. The present invention can generate a facial image of an individual by selecting the facial feature part shapes that match the individual based on the identifying composite code for the individual. Accordingly, an increasingly larger number of identifying xe2x80x9ccompositexe2x80x9d codes can be stored locally, without requiring substantial additional memory.
The present invention also has the ability to recognize a suspect""s facial image by assigning identifying composite code values to the image, and then using these codes to select mug shot photographs having the same and similar identifying code values. The shape of each facial feature part is identifiable by a low spatial resolution 3-D technique that uses a relatively small number, (e.g., less than fifty (50) points and, in the preferred embodiment is only 32 points), facial feature fiducial points. These fiducial points are assigned to anatomically critical points of a facial feature part. A fiducial point is a point on the surface of a facial feature part that can be used to identify the shape of the particular part and to assign a proper composite code to the facial feature. Each fiducial point is specified by a position vector that defines a position of the surface point. The position vector may be defined by only thirty (30) bits of data. Each facial feature shape may be identified by 32 position vectors that locate the positions of surface features of the shape of the facial part. The fiducial points are used to identify a facial image and to compare the image of one face to other facial images. By identifying the fiducial points on the picture of a suspect, the suspect""s facial image may be mapped to the 3-D facial feature parts and used to determine the composite codes for the facial image and generate a 3-D facial image of the suspect. In addition, once the composite codes may be used to search for other facial images that are the same as or similar to the suspect""s facial image. Accordingly, the invention has the capability of facial image recognition using a relatively small number of facial feature fiducial points.
The invention also provides a fast search engine for finding facial images that are the same or similar to the facial image of a suspect or other person. By mapping the fiducial points of a facial image of an individual, the composite codes corresponding to the individual""s facial image can be determined. These composite codes are used to search for other facial images (in mug shot databases) that have the same or similar composite codes. The search for the same and similar composite codes in tens of millions of mug shot codes is done much faster than would an image-to-image search through the mug shot images. Thus, the present invention allows for a quick search to be conducted of other mug shots that match the image of a suspect.